In general, the present invention relates to a picture editing apparatus and a picture editing method. More particularly, the present invention relates to an excellent picture editing apparatus and an excellent picture editing method used for example for carrying out the so-called cut/edit processing on a bit stream typically obtained from an MPEG encoding operation of a picture.
FIG. 11 is a block diagram showing a typical configuration of the conventional picture editing apparatus.
Implemented typically by a hard disc, a data storage unit 21 employed in the picture editing apparatus shown in the figure is used for storing bit streams A and B as objects to be edited. The bit streams A and B have been compressed in conformity with the ISO-11172-2 specifications or the so-called MPEG1 specifications. The data storage unit 21 is also used for storing, among other data, a continuous bit stream C resulting from editing of the bit streams A and B in accordance with commands issued by a controller 124. To put it in detail, the bit stream C is a result of concatenation of at least a portion extracted from the bit stream A and at least a portion from the bit stream B with each other.
A bit counter 22 is used for counting the number of bits included in a bit stream output from the data storage unit 21. The number of bits included in a bit stream is used for expressing the size of the bit stream. The number of bits is supplied to the controller 124 at a request made by the controller 124. In addition, the bit counter 22 also passes on a bit stream received from the data storage unit 21 to a decoder 23 as it is. Controlled by the controller 124, the decoder 23 decodes a variety of parameters included in a bit stream supplied thereto from the data storage unit 21 by way of the bit counter 22 and outputs results of the decoding to the controller 124. To put it in detail, the decoder 23 is used for decoding headers included in a bit stream such as a sequence header, a GOP header, that is, the header of a GOP (Group of Pictures), and a picture header, supplying information recorded in the headers to the controller 124.
On the basis of data output by the bit counter 22 and the decoder 23, the controller 124 composes a bit stream C from the bits streams A and B stored in the data storage unit 21 by using a memory unit 25 if necessary and stores the bit stream C in the storage unit 21.
The operation of the picture editing apparatus to compose the bit stream C is explained as follows.
Assume that the bit stream C is composed by concatenating a scene A' extracted from the bit stream A with a scene B' cut out from the bit stream B with the scenes A' and B' serving as the front and rear halves of the concatenation respectively as shown in FIG. 12.
Also assume that the start and end points of the scene A' are A.sub.in and A.sub.out respectively whereas the start and end points of the scene B' are B.sub.in and B.sub.out respectively.
According to the MPEG1 specifications, a bit stream shall be edited in GOP units. Thus, the start points A.sub.in and B.sub.in are each set in advance to point to a head picture of a GOP while the end points A.sub.out and B.sub.out are each set in advance to point to a tail picture of a GOP.
In order to detect the start points A.sub.in and B.sub.in as well as the end points A.sub.out and B.sub.out, it is therefore necessary to detect their GOPs. A GOP is normally detected by recognition of a GSC (Group Start Code) placed at the head of the GOP. In the case of the last GOP at the end of the bit stream A with no GOP following it, the end point A.sub.out thereof is detected by assuming that another GOP follows the last GOP of the bit stream A. By the same token, for the last GOP of the bit stream B, the end point B.sub.out thereof is detected by assuming that another GOP follows the last GOP.
FIG. 13 is a flowchart representing operations carried out by the picture editing apparatus to concatenate the scene A' with the scene B'. As shown in the figure, the flowchart begins with a step S101 at which sequence headers are read out from the bit streams A and B. To put it in detail, the controller 124 controls the data storage unit 21 so as to supply the bit stream A to the decoder 23 by way of the bit counter 22. The decoder 23 then decodes the sequence header of the bit stream A. Subsequently, the same operations are carried out on the bit stream B.
The decoded contents of the sequence headers of the bit streams A and B are supplied to the memory unit 25 by way of the controller 124 to be stored therein.
Then, the flow of the processing goes on to a step S102 at which the controller 124 compares the sequence header of the bit stream A stored in the memory unit 25 with that of the bit stream B in order to form a judgment as to whether or not cut/edit processing can be carried out on the bit streams A and B to compose a bit stream C. To put it in detail, if parameters up to a LIQM (Load Intra Quantize Matrix) parameter of the sequence header of the bit stream A do not agree with those of the bit stream B, the cut/edit processing can not be carried out. Thus, at the step S102, the parameters of the bit stream A are compared with those of the bit stream B to determine whether or not the cut/edit processing can be carried out.
If the outcome of the judgment formed at the step S102 indicates that the cut/edit processing can not be carried on the bit streams A and B, that is, if even only one of the parameters up to the LIQM parameter of the sequence header of the bit stream A does not agree with the corresponding parameter of the bit stream B, the controller 124 displays a message indicating that the processing can not be carried out on a monitor not shown in the figure and terminates the processing.
If the outcome of the judgment formed at the step S102 indicates that the cut/edit processing can be carried on the bit streams A and B, that is, if the parameters up to the LIQM parameter of the sequence header of the bit stream A all agree with the corresponding ones of the bit stream B, on the other hand, the flow of the processing continues to a step S103 at which the controller 124 starts an operation to read out the bit stream A from the data storage unit 21 typically in picture units, supplying the bit stream A to the decoder 23 by way of the bit counter 22.
In the bit counter 22, the number of bits included in the bit stream A representing the size of the bit stream A is counted whereas, in the decoder 23, the sequence header of the bit stream A, if any, is decoded. Results of the counting and the decoding are both supplied to the controller 124. Then, the flow of the processing proceeds to a step S104 at which the controller 124 forms a judgment as to whether or not a sequence header was output from the data storage unit 21 by monitoring the result of the decoding supplied thereto by the decoder 23. If the outcome of the judgment formed at the step S104 indicates that a sequence header was output, the flow of the processing continues to a step S105 at which the most recent sequence header supplied by the decoder 23 is used to update an existing sequence header for the bit stream A stored in the memory unit 25. The flow of the processing then goes on to a step S106.
If the outcome of the judgment formed at the step S104 indicates that a sequence header was not output, on the other hand, the flow of the processing continues directly to the step S106 by skipping the S105. At the step S106, the controller 124 forms a judgment as to whether or not a picture at the start point A.sub.in was read out from the data storage unit 21. If the outcome of the judgment formed at the step S106 indicates that a picture at the start point A.sub.in has not been read out yet, the flow of the processing returns to the step S103 at which the controller 124 continues the operation to read out the bit stream A from the data storage unit 21.
If the outcome of the judgment formed at the step S106 indicates that a picture at the start point A.sub.in has been read out, on the other hand, the flow of the processing proceeds to a step S107 at which the controller 124 stores the most recent sequence header of the bit stream A stored in the memory unit 25 in the data storage unit 21 as the first sequence header of a bit stream C currently being composed. The flow of the processing then goes on to a step S108. At the step S108, the controller 124 reads out data of the GOP header of a GOP, to which the picture at the start point A.sub.in pertains, from the data storage unit 21. The controller 124 then requests the decoder 23 to decode the data and receives a result of the decoding from the decoder 23.
Then, the flow of the processing continues to a step S109 at which the controller 124 forms a judgment as to whether or not a CG (closed GOP) field included in the GOP header received from the decoder 23 is 0. If the outcome of the judgment formed at the step S109 indicates that the CG field is 0, that is, if the GOP is a so-called open GOP, the flow of the processing goes on to a step S110 at which a BL (Broken Link) field included in the GOP header is set at 1. The flow of the processing then proceeds to a step S112.
This is because, if the current GOP including a picture at the start point A.sub.in is an open GOP, the link to the immediately preceding GOP is broken by the edit processing making it impossible to correctly decode a first B picture in the display order of the current GOP in a playback operation. That is why the BL field is set at 1 to indicate the fact that the link to the immediately preceding GOP is broken.
If the outcome of the judgment formed at the step S109 indicates that the CG field is not 0, that is, if the GOP is a so-called closed GOP, on the other hand, the flow of the processing goes on to a step S111 at which the BL (Broken Link) field included in the GOP header is set at 0. The flow of the processing then proceeds to the step S112.
This is because, if the current GOP including a picture at the start point A.sub.in is a closed GOP, the first picture of the current GOP is an I picture or a B picture requiring no forward direction prediction. In either case, the immediately preceding GOP is not required in the decoding of the picture. That is why the BL field is set at 0.
At the step S112, the controller 124 stores the GOP header with the BL field thereof set at the step S110 or S111 as a GOP header of the bit stream C in the data storage unit 21 following the sequence header stored at the step S107. The flow of the processing then goes on to a step S113.
At the step S113, the controller 124 continues the operation to read out the bit stream A from the data storage unit 21. Then, the flow of the processing proceeds to a step S114 at which the controller 124 forms a judgment as to whether or not a picture at the end point A.sub.out has been read out from the data storage unit 21. If the outcome of the judgment formed at the step S114 indicates that a picture at the end point A.sub.out has not been read out, the flow of the processing continues to a step S115 at which data read out at the step S113 is stored in the data storage unit 21 as data of the bit stream C.
Subsequently, the flow of the processing goes on to a step S116 at which the controller 124 forms a judgment as to whether or not the data read out at the step S113 is a sequence header. If the outcome of the judgment formed at the step S116 indicates that the data read out at the step S113 is a sequence header, the flow of the processing continues to a step S117 at which the most recent sequence header supplied by the decoder 23, that is, the data read out at the step S113, is used to update an existing sequence header for the bit stream A stored in the memory unit 25. The most recent sequence header of the bit stream A will be compared with the sequence header of the bit stream B at a step S128 to be described later. The flow of the processing returns to a step S113 at which the controller 124 continues the operation to read out the bit stream A from the data storage unit 21.
If the outcome of the judgment formed at the step S116 indicates that the data read out at the step S113 is not a sequence header, on the other hand, the flow of the processing returns directly to the step S113 to continue the operation to read out the bit stream A from the data storage unit 21 by skipping the step S117.
If the outcome of the judgment formed at the step S114 indicates that a picture at the end point A.sub.out has been read out, on the other hand, the flow of the processing continues to a step S121 shown in FIG. 14, a diagram showing the continuation to the flowchart shown in FIG. 13. At the step S121, data read out at the step S113 is stored in the data storage unit 21 as data of the bit stream C. The flow of the processing then goes on to a step S122 at which the controller 124 continues the operation to read out the bit stream A from the data storage unit 21. Then, the flow of the processing proceeds to a step S123 at which the controller 124 forms a judgment as to whether or not data read out at the step S122 is either a sequence header, a GOP header or an SEC (Sequence End of Code) by referring to an output of the decoder 23.
If the outcome of the judgment formed at the step S123 indicates that the data read out at the step S122 is neither a sequence header, a GOP header nor an SEC, the flow of the processing returns to the step S121 to repeat the processing carried out at the steps S121 to S123.
If the outcome of the judgment formed at the step S123 indicates that the data read out at the step S122 is either a sequence header, a GOP header or an SEC, on the other hand, the flow of the processing goes on to the step S124 without storing the data in the data storage unit 21. At the step S124, the controller 124 starts an operation to read out the bit stream B from the data storage unit 21 typically in picture units.
Then, the flow of the processing proceeds to a step S125 at which the controller 124 forms a judgment as to whether or not data read out at the step S124 is a sequence header by referring to an output of the decoder 23. If the outcome of the judgment formed at the step S125 indicates that a sequence header was read out, the flow of the processing continues to a step S126 at which the most recent sequence header supplied by the decoder 23 is used to update an existing sequence header for the bit stream B stored in the memory unit 25. The flow of the processing then goes on to a step S127.
If the outcome of the judgment formed at the step S125 indicates that the data read out at the step S124 is not a sequence header, on the other hand, the flow of the processing continues directly to the step S127 by skipping the S126. At the step S127, the controller 124 forms a judgment as to whether or not a picture at the start point B.sub.in was read out from the data storage unit 21. If the outcome of the judgment formed at the step S127 indicates that a picture at the start point B.sub.in has not been read out yet, the flow of the processing returns to the step S124 at which the controller 124 continues the operation to read out the bit stream B from the data storage unit 21.
If the outcome of the judgment formed at the step S127 indicates that a picture at the start point B.sub.in has been read out, on the other hand, the flow of the processing proceeds to a step S128 at which the controller 124 compares the most recent sequence headers of the bit streams A and B stored in the memory unit 25 with each other to form a judgment as to whether or not the LIQM parameters of the sequence headers match each other and as to whether or not the LNIQM (Load Non Intra Quantizer Matrix) parameters of the sequence headers agree with each other.
If the outcome of the judgment formed at the step S128 indicates that the LIQM and/or the LNIQM parameters of the most recent sequence headers do not agree with each other, the flow of the processing goes on to a step S129 at which the controller 124 stores the most recent sequence header for the bit stream B stored in the memory unit 25 in the data storage unit 21 as a sequence header of the bit stream C currently being composed. The flow of the processing then proceeds to a step S130.
If the outcome of the judgment formed at the step S128 indicates that the LIQM and the LNIQM parameters of the most recent sequence headers agree with each other, on the other hand, the flow of the processing directly goes on to the step S130 by skipping the step S129. At the step S130, the controller 124 computes a VD (VBV Delay) for a picture of the start point B.sub.in read out at the step S124. The VD represents the amount of data to be accumulated in a VBV (video Buffering Verifier) buffer assumed to exist on the receiving side, that is, on the decoder side, as prescribed in the MPEG specifications.
The VD and its computation at the step S130 will be described more later.
Then, the flow of the processing proceeds to a step S131 at which the controller 124 forms a judgment as to whether a value of the VD computed at the step S130 is proper or improper, the meanings of which will be described more later. An outcome of the judgment formed at the step S131 showing an improper value of the VD computed at the step S130 indicates that the bit streams A and B can not be edited prior to MPEG decoding. In this case, the processing is thus terminated.
If the outcome of the judgment formed at the step S131 indicates that the value of the VD computed at the step S130 is not improper, on the other hand, the flow of the processing goes on to a step S132 at which the value of the VD computed at the step S130 is written over a VD field in the picture header of the picture of the start point by read out at the step S124. The flow of the processing then continues to a step S133 at which the picture with the VD thereof updated at the step S132, that is, the picture of the start point B.sub.in read out at the step S124 in this case, is stored in the data storage unit 21 as data of the bit stream C being composed.
Then, the flow of the processing proceeds to a step S134 at which the controller 124 continues the operation to read out the bit stream B from the data storage unit 21. Subsequently, the flow of the processing proceeds to a step S135 at which the controller 124 forms a judgment as to whether or not a picture at the end point B.sub.out has been read out from the data storage unit 21. If the outcome of the judgment formed at the step S135 indicates that a picture at the end point B.sub.out has not been read out at the step S134, the flow of the processing returns to a step S130 to repeat the processing carried out at the steps S130 to S135.
To be more specific, at the step S130, the controller 124 computes a VD, which represents to the amount of data to be accumulated in the VBV buffer, for a picture read out at the step S134. Then, the flow of the processing proceeds to the step S131 at which the controller 124 forms a judgment as to whether a value of the VD computed at the step S130 is proper or improper. If the outcome of the judgment formed at the step S131 indicates that value of the VD computed at the step S130 is improper, the processing is terminated. If the outcome of the judgment formed at the step S131 indicates that the value of the VD computed at the step S130 is not improper, on the other hand, the flow of the processing goes on to the step S132 at which the value of the VD computed at the step S130 is written over a VD field in the picture header of the picture read out at the step S134. The flow of the processing then continues to the step S133 at which the picture with the VD thereof updated at the step S132, that is, the picture read out at the step S134 in this case, is stored in the data storage unit 21 as data of the bit stream C being composed.
If the outcome of the judgment formed at the step S135 indicates that a picture at the end point B.sub.out has been read out at the step S134, on the other hand, the flow of the processing goes on to a step S136 at which the controller 124 stores the picture, that is, the picture of the end point B.sub.out read out at the step S134 in this case, in the data storage unit 21 as data of the bit stream C being composed. It should be noted that the processing carried out at the step S136 includes the pieces of processing carried out at the steps S130 to S132, that is, the computation of a value of the VD, the formation of the judgment as to whether the computed value of the VD is proper or improper and the updating of the VD field in the picture header in the case of a proper computed value of the VD.
Later on, the flow of the processing proceeds to a step S137 at which an SEC is stored as last data of the bit stream C being composed and the processing is terminated.
By the way, the MPEG specifications prescribe that a bit stream shall be composed so that neither an overflow nor an underflow will occur in the VBV buffer.
FIG. 15 is a diagram showing changes of the amount of data accumulated in the VBV buffer. It should be noted that the horizontal and vertical axes of the diagram represent time and the amount of accumulated data respectively.
The VBV buffer is used for accumulating a bit stream resulting from MPEG encoding at a predetermined fixed bit rate. A sequence header of a bit stream includes a BR (Bit Rate), a value resulting from division of the fixed bit rate expressed in terms of bps (bits per second) by 400 with fraction part of the value rounded up.
A bit stream is read out from the VBV buffer to be supplied to a decoding unit in accordance with a PR (Picture Rate) included in the sequence header of the bit stream. To put it in detail, at each of decoding times which occur at fixed intervals determined by the PR, data of a picture to be decoded at the decoding time is read out instantaneously from the VBV buffer and supplied to the decoding unit.
Thus, during a period between an operation to read out a picture from the VBV buffer and an operation to read out a next picture, data of the amount of a bit rate/a picture rate is accumulated in the VBV buffer. Notation D.sub.1 shown in FIG. 15 represents the amount of data equal to a bit rate/a picture rate.
At a decoding time, data starting with a first bit of an SHC (Sequence Header Code), a GSC (Group Start Code) or a PSC (Picture Start Code) and ending with a bit immediately preceding the next SHC, the next GSC or the next PSC is read out instantaneously from the VBV buffer. The SHC is a code indicating the start of a sequence header. According to the MPEG1 specifications, the SHC has a hexadecimal value of 000001B3. On the other hand, the GSC is a code indicating the start of a GOP header. According to the MPEG1 specifications, the GSC has a hexadecimal value of 000001B8. Finally, the PSC is a code indicating the start of a picture header. According to the MPEG1 specifications, the PSC has a hexadecimal value of 00000100. It should be noted that notation D.sub.1 shown in FIG. 15 represents the amount of data starting with a first bit of an SHC and ending with a bit immediately preceding the next SHC.
The amount of data accumulated in the VBV buffer can be recognized from the VD included in a picture header. The VD expresses the number of clock pulses with a frequency of 90 KHz resulting from measurement of the length of a time expressed in terms of clock pulses. The time is a period of time it takes to restore the amount of data accumulated in the VBV buffer in a process to accumulate data in the VBV buffer at a bit rate indicated by the BR following an operation to read out data up to the last bit of a PSC from the VBV buffer to a value prior to the read operation.
In an edit operation described above, normally, there is a gap representing a difference between the amount of data accumulated in the VBV buffer at the end point A.sub.out of the bit stream A and the amount of data accumulated in the VBV buffer at the start point B.sub.in of the bit stream B as shown in FIG. 16, a diagram showing a mismatch of the amount of data accumulated in the VBV buffer which is resulted in when the scene A' of the bit stream A is concatenated with the scene B' of the bit stream B. In order to close the gap, it is thus necessary to modify the VD of a picture at the start point B.sub.in so that the amount of data accumulated in the VBV buffer at the start point B.sub.in of the bit stream B matches the amount of data accumulated in the VBV buffer at the end point A.sub.out of the bit stream A.
To put it concretely, it is necessary to change the VD of a picture at the start point B.sub.in to VD' (B.sub.in) expressed by the following equation : EQU VD'(B.sub.in)=VD(A.sub.out) EQU +(HL (B.sub.in)-HL(A.sub.out)) EQU .times.90000/br (1)
where a notation VD (A.sub.out) in the above equation (1) is the VD of a picture at the end point A.sub.out whereas a symbol br is a bit rate found from the BR. Notation HL (B.sub.in) denotes the number of bits, from the first bit of a sequence header or a GOP header to the last bit of a PSC, of a picture at the start point B.sub.in, and notation HL (A.sub.out) denotes the number of bits, from the first bit of a sequence header or a GOP header to the last bit of a PSC, of a picture at the end point A.sub.out.
Thus, at the step S130 of the flowchart shown in FIG. 14, the VD for a picture at the start point B.sub.in, that is, VD' (B.sub.in), is computed in accordance with Eq. (1).
If the VD of a picture at the start point B.sub.in is changed to VD' (B.sub.in) found by Eq. (1), it is also necessary to change subsequent VDs included in the bit stream B. The subsequent VDs can be changed by adding VD' (B.sub.in)-VD (B.sub.in) to the original values thereof where VD (B.sub.in) is the value of the VD for the picture at the start point B.sub.in prior to the change according to Eq. (1).
At the step S130 of the flowchart shown in FIG. 14, the VD for a picture other than that at the start point B.sub.in is found as follows.
It should be noted a value of the VD after being changed of 0 or smaller results in an underflow. On the other hand, a value of the VD after being changed greater than 90000.times.S.sub.VBV /br or greater than the hexadecimal FFFE causes an overflow. Since neither an underflow nor an overflow is allowed to occur in the VBV buffer, such a negative or excessively large value of the VD is not allowed. Therefore, at the step S131 of the flowchart shown in FIG. 14, the controller 124 forms a judgment as to whether or not the value of the VD computed at the step S130 is equal to or smaller than 0 or excessively large.
By the way, in the conventional edit processing described above, it is basically necessary to change all the VDs of pictures at the start point B.sub.in and the subsequent pictures, giving rise to a hindrance to an effort to increase the speed of the processing.
In addition, an improper value of the VD described above makes it impossible to edit the bit streams prior to MPEG decoding, causing the editing work to become troublesome.
Moreover, the bit rate br which is required in the computation of the VD is found from a BR included in a sequence header. As described above, however, the BR is a value resulting from division of the true bit rate expressed in terms of bps (bits per second) by 400 with fraction part of the value rounded up. Thus, in some cases, the VD can not be found with a high degree of accuracy.